Performance Metrics Cheat Sheet




\(\begin{aligned} {\text{Prevalence}} = \frac{\text{TP + FN}}{\text{TP + FP + TN + FN}} \end{aligned}\)


\(\begin{aligned} \text{Sensitivity (Recall, True Positive Rate)} = \frac{\text{TP}}{\text{TP + FN}} = \frac{\text{TP}}{\text{Real Positives}} = \text{Prob( Predicted Positive | Real Positive )} \end{aligned}\)


\(\begin{aligned} \text{Specificity (True Negative Rate)} = \frac{\text{TN}}{\text{TN + FP}} = \frac{\text{TN}}{\text{Real Negatives}} = \text{Prob( Predicted Negative | Real Negative )} \end{aligned}\)


\(\begin{aligned} \text{PPV (Precision)} = \frac{\text{TP}}{\text{TP + FP}} = \frac{\text{TP}}{\text{Predicted Positives}} = \text{Prob( Real Positive | Predicted Positive )} \end{aligned}\)


\(\begin{aligned} \text{NPV} = \frac{\text{TN}}{\text{TN + FN}} = \frac{\text{TN}}{\text{Predicted Negatives}} = \text{Prob( Real Negative | Predicted Negative )} \end{aligned}\)


\(\begin{aligned} \text{Lift} = \frac{\text{PPV}}{\text{Prevalence}} = \frac{\cfrac{\text{TP}}{\text{TP + FP}}}{\cfrac{\text{TP + FN}}{\text{TP + FP + TN + FN}}} \end{aligned}\)


\(\begin{aligned} \text{Net Benefit} = \frac{\text{TP}}{\text{TP + FP + TN + FN}} - \frac{\text{FP}}{\text{TP + FP + TN + FN}} * {\frac{{p_{t}}}{{1 - p_{t}}}} \end{aligned}\)




Calibration

Smooth

## # A tibble: 30 x 4
##    model quintile  phaty     phatx
##    <fct>    <int>  <dbl>     <dbl>
##  1 First        1 0      0.0000975
##  2 First        2 0      0.000979 
##  3 First        3 0      0.00499  
##  4 First        4 0      0.0148   
##  5 First        5 0.0667 0.0629   
##  6 First        6 0.133  0.178    
##  7 First        7 0.533  0.405    
##  8 First        8 0.867  0.791    
##  9 First        9 0.733  0.899    
## 10 First       10 1      0.976    
## # ... with 20 more rows
## [1] -0.05  1.05

Discrete

## # A tibble: 30 x 4
##    model quintile  phaty     phatx
##    <fct>    <int>  <dbl>     <dbl>
##  1 First        1 0      0.0000975
##  2 First        2 0      0.000979 
##  3 First        3 0      0.00499  
##  4 First        4 0      0.0148   
##  5 First        5 0.0667 0.0629   
##  6 First        6 0.133  0.178    
##  7 First        7 0.533  0.405    
##  8 First        8 0.867  0.791    
##  9 First        9 0.733  0.899    
## 10 First       10 1      0.976    
## # ... with 20 more rows
## [1] -0.05  1.05

Probability Threshold Dependent Performance Metrics

By Probability Threshold

Performance Metrics Curves

ROC

Lift

Precision Recall

Gains

Decision

Performance Table

By Predicted Positives Condition Rate (PPCR)

Performance Metrics Curves

ROC

Lift

Precision Recall

Gains

Decision

Performance Table